Visualizable and Interpretable Regression Models With Good Prediction Power

Many methods can fit models with higher prediction accuracy, on average, than least squares linear regression. But the models, including linear regression, are typically impossible to interpret or visualize. We describe a tree-structured method that fits a simple but non-trivial model to each partition of the variable space. This ensures that each piece of the fitted regression function can be visualized with a graph or a contour plot. For maximum interpretability, our models are constructed with negligible variable selection bias and the tree structures are much more compact than piecewise-constant regression trees. We demonstrate, by means of a large empirical study involving twenty-seven methods, that the average prediction accuracy of our models is almost as high as that of the most accurate “black-box” methods from the statistics and machine learning literature.